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(54) Image sequence coding and decoding method 

(57) In order to prevent accumulation of rounding 
enrors caused by the integer-based bilinear interpolation 
used in the motion compensation process of image cod- 
ing and decoding methods, two types of P frames are . 
used for unidtrectk>nal motion compensation predk;tion, 
namely: 



P+ frames, which round the results of bilinear inter- 
polation with real number operatiorrs to the nearest 
integer and rounds haH integer values (0.5 added to 
an integer) away from zera and 
P- frames, which differs from p4 frames in that the 
above menfkHied half integer values are rounded 
towards zera 

Utilising both of these P-i- and P- frames enables cancel- 
ling of rounding errors and prevents the accumulatkm of 
rounding errors. 
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Description 

BACKGROUND OF THE INVENTION 
5 Field of the Invention 

The present invention relates to an image sequence cocfing and decoding method virhich performs interlrame pre- 
diction using quantized values for chrominance or luminance intensity. 

10 Descripton of Related Art 

In high efficiency coding of image sequences, interframe prediction (motion compensation) by utilizing the similarly 
of adjacent frames over time, is knov^n to be ahighiy effective technique for data compression. Today's most frequently 
used motion compensation method is block matching virith half pixel accuracy, which is used in interrmtional standards 
16 H.263, MPEG1, and MPEG2. In this method, the inriage to be coded is segmented into blocks and the horizontal and 
vertical components of the rrx>tion vectors of these blocks are estimated as integral multiptes of half the distance 
between adjacent pixels. This process is described using the following equation: 
[Equatkxil] 

20 P(x,y)-/?(x+i;/.y+v,(x.y)€B,.Oi/<A/ (1) 

where P{x, y) and R(x, y) denote the sample values (luminance or chrominarwe intensity) of pixels located at coordi- 
nates (x. y) in the predicted image P of the current frame arvj the reference image (decoded image of a frarne which 
has been encoded before the current frame) R. respectively x and y are integers, and H is assumed that all the pixels 

25 are located at points v^ere the coordinate values are integers. Additionally it is assumed that the sample values of the 
pixels are quantized to non-negative integers. N. Bi. and (ui. vi] derx^te the number of blocks in the image, the set of 
pixels induded in the i-th block of the image, and the motion vectors of the i-th block, respectively. 

When the values for ui and vi are not integers, it is necessary to find the intensity value at the point where no pix^ 
actually exist in the reference image. Currently, bilinear interpolation using the adjacent lour pixels is the most frequently 

30 used method for this process. This interpolation method is described using the foltowing equation: 
[Equation q 

fl(x+g,y+9^((d.q)((d-p)«(jr.y)+pfl{jf+1.y)) +Q((d-p)fl(x.y+1)+pfl(;r+l.y+l))y/d^ (2) 

35 

wh&e d is a positive integer, and p and q are smaller than d but not smaller than 0. Vr denotes Integer diviston which 
rounds the result of normal division (division using real nun^)ers) to the nearest integer. 

An example of the structure of an H.263 video encoder is shown in Fig. 1. As the coding algorithm, H.263 adopts 
a hytxid coding method (adaptive interf rame/intraframe coding method) whk:h is a combination of block matching and 

40 DCT (discrete cosine trartsform). A subtracter 102 calcinates the difference between the input image (current frame 
base image) 101 and the output image 113 (related later) of the interframe/intraframe coding selector 119 , and then 
outputs an error image 1 03. This error image is quantized in a quantizer 1 05 after being converted into DCT coeff k^tents 
in a DCT converter 104 and then forms quantized DCT coeffidents 106. These quantized DCT coefficients are trans- 
mitted through the communication channel while at the same time used to synthesize the interframe predicted image in 

45 the encoder. The procedure for synthesizing the predk:ted image is explained next The abosfe mentioned quantized 
DCT coefficients 106 forms the reconstructed error image 110 (same as the reconstructed error image on the receive 
side) after passing through a dequantizer 108 and inverse DCT converter 109. This reconstructed error image and the 
output image 1 13 of the interframe /intraframe coding selector 119 is added at the adder 1 11 and the decoded image 
1 12 of the current frame (sanie image as the decoded image of current frame reconstructed on the receiver side) is 

50 obtained. This image is stored in a frame memory 1 14 and delayed for a time equal to the frame intend. Aooordingly. 
at the current point the frame meniory 1 14 outputs the decoded image 1 1 5 of the previous frame. This decoded Image 
of the previous frame and the original image 101 of the cunrent frame are input to the block matching section 116 and 
block matching isperformed betvtreen these images In the bk>ck matching process, the original image of the current 
frame is segmented into multiple blocks, and the predicted image 1 1 7 of the cunent frame is synthesized by extracting 

55 the section most resembling these blocks from tine decoded image of the previous frame. In ttiis process, it is necessary 
to estimate the motion between the prkjr frame and ttie curent frame for each block. TTie motion vector tor each bkxk 
estimated in the motion estimation process is transmitted to the receiver side as motion vector data 1 20. On the receiver 
side, the same prediction image as on the transmitter side is synthesized using the motk)n vector infonnation and the 
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decoding image of the previous frame. The prediction innage 1 17 Is input along with a "O* signal 1 1B to the interfranne 
/intraframe coding selector 119. This switch 119 selects interframe coding or intraframe coding by selecting eittwof 
these Inputs. Interframe coding is perlormed when the prediction Image 117 is selected {XHt case Is shown In Fig. 2). 
On the other hand when the "0" signal is selected, intraframe coding is perfomned since the input inriage itself is oon- 
s verted, to a OCT coefficients and output to the communication channel. In order for the receiver side to correctly recorv 
struct the coded image, the reciever must be informed whether intraframe coc£ng or interframe coding was performed 
on the transmitter side. Consequently, an identifier flag 121 Is output to the communication circuit Rnalty, an H.263 
coded bitstream 123 is acquired by multiplexing the quantized DCT coefficients, motion vectors, the and inter- 
frameAntraframe identifier ftag information in a multiplexer 122. 

10 The structure of a decoder 200 for receiving the coded bit stream output from the encoder of Fig. 1 is shown in Rg. 
2. The H.263 coded bit stream 21 7 that is received is derrultiplexed Into quantized DCT coefficients 201 . motion vector 
data 202. and a interlrame/intraframe identifier flag 203 in the demultiplexer 216. The quantized DCT coefficients 201 
become a decoded error image 206 after being processed by an inverse quantizer 204 and inverse DCT converter 205. 
This decoded error image is added to the output image 21 5 of the interframe /intraframe coding selector 214 in an adder 

16 207 and the sum of these images is output as the decoded image 208. The output of the interframe /intraframe coding 
selector is switched according to the interframe/intraframe identifier flag 203. A prediction image 212 utilized when per* 
forrr^ng interframe encoding is synthesized in the prediction image synthesizer 21 1 . In this synthesizer, the position of 
the blocks in the decoded image 210 of the prior frame stored in frame memory 209 is shrfled according to the motion 
vector data 202. On the other hand, for intraframe coding, the interframe /intraframe coding selector outputs the ^O" sig* 
nal213asis. 

SUMI^ARY OF THE INVEfsfTION 

The image encoded by H.263 is comprised of a luminance plane (Y plane) containing tuminarice information, and 

25 two chrominance planes (U plane and V plane) containing chrominance information. At this time. characteristica(ly. 
when the image has 2m pixels in the horizorrtal direction and 2n pixels in the vertical direction (m arxl n are positive Inte- 
gers), the Y plane has 2m pixels horizontally and 2n pixels vertically, the U and V planes have m pixels horlzontaOy and 
n pixels vertically. The low resolution on the chrominance plane is due to the tect that the human visual system has a 
comparatively dull visual faculty with respect to spatial variations in chrominance. Having such image as an input H. 

30 263 performs coding and decoding in t>lock units referred to as macroblocks. The structure of a macroblock is shown in 
Fig. 3. The nnacroblock is comprised of three blocte; a Y blocK U block arxl V block. The size of the Y block 301 con- 
taining the luminance information is 16 X 16 pixels, and the size of the U YjHock 302 and V bk)ck 303 containing the 
chrominance infbrmatk>n is 8 X 8 pixels. 

In H. 263. half pixel accuracy block matching is applied to each t>lock. Accordingly, when the estimated motion vec- 

35 tor is defined as (u. v), u and v are both integral multiples of half the distarKe between pixels. In other words, 1/2 is used 
as the minimum unit. The configuration of the interpolation method used for the intensity values (hereafter the intensity 
values for "luminance" and "chrominance" are called by the general term "intensity value") is shown in Rg. 4. When per- 
forming the interpolation described in equation 2. the quotients of division are rounded off to the nearest integer, and 
further, when the quotient has a half integer value (i.e. 0.5 added to an integer), rounding off is performed to the next 

40 integer in the direction away from zero. In other words, in Rg. 4, when the intensity values for 401, 402, 403, 404 are 
respectively La. Lb. Lc, and Ld (La, Lb, Lc, and Ld are non-negative integers), the interpolated intensity values la, Ic, 
and Id (la, lb. lc, and td are non-negative integers) at positions405. 406. 407, 408 are expressed by the following equa- 
tion: 

[Equation 3] 

46 

iatMLa (3) 

60 tcm[{L3^LC'¥^y^] 

ld^[{LafLh'¥U*Ld-*2)/4] 

where 1 )" denotes truncation to the nearest integer towards 0 (i.e. the fractional part is discarded). The expectation of 
55 the errors caused by this rounding to integers is estimated as follows: ft is assumed that the probability that the Intercity 
value at positions 405, 406. 407. and 408 of Rg. 4 is used is all 25 percent When finding the intensity value la for posi- 
tion 405, the rounding error will dearly be zero. Also, when finding the intensity value ft) for position 406, the error vmII 
be zero when La-i-Lb is an even number, and when an odd nunt>er the error is 1/2. If the probabiFity that La+Lb wfll be 
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an even nun^er and an odd number is both 50 percent, then the expectation for the error will be 0x1/2 •••1/2X1/ 
2 « 1 / 4. Further, when finding the intensity value Ic for position 407, the expectation fa the error Is 1 / 4 as for b. Wh«n 
finding the intensity value Id for position 408. the error when the residual of La-f LiHlx-t-Ld divided by four are 0, 1« 2, 
and 3 are respectively 0. -1/4. 1/2. and 1/4. If we assume that the probability that the residual is 0. 1 , 2. and 3 is all equal 

5 (i.e. 25 percent), the expectation for the error Is 0x1/4-1/4x1/441/2x1/4+1/4x1/4 - I / 8; As descrbed 
above, assuming that the possbility that the intensity value at positions 405 - 408. being used are all equal, the final 
expectation for the error i&0x1/44l/4x1/4+1/4x1/4 + 1/ 8x1/4-5/32.Tbis indicates that each time 
nx)tion compensation is performed by means of block nnatchtng. an error of 5/32 occurs in the pixel intensity value. Gen- 
erally in low rate coding, sufficient number of bits cannot t>e used for the encoding of the interframe error difference so 

10 that the quantized step size of the DCT coefficient is prone to be large. Accordingly, errors occunirtg due to motion com- 
pensation are corrected only when it is very large. When interframe encoding is performed continuously wittxxJt per- 
forming intralrame coding under such enviraiment. the errors tend, to accumulate and cause bad effects on the 
reconstructed image. 

Just as explained above, the number of pixels is about half in both the vertical and horizontal direction on the 

16 chrominance plane Therefore, for the motion vectors of the U btock and V block, half the value of the motion vector for 
the Y block is used for the vertical and horizontal components. Since the horizontal and vertical components of the 
motion vector for the Y block motion vector are integral multiples of 1/2. the motion vecta components for the U arvj V 
blocks will appear as integral multiples of 1/4 (quarter pixel accuracy) if ordinary division is implemented. However, due 
to the high oomputatbnal complexity of the intensity Interpolation process for motion vectors with quarter pixel aocu- 

20 racy, the motion vectors for U and V blocks are rounded to half pixel accuracy in H.263, The rounding method utilized 
in H.263 is as follows: According to the definition described akx3ve. (u, v) denotes the motion vector of the maaxWock 
(which is equal to the nr\otion vector for the Y block). Assuming that r is an integer arxj s is an non-negative integer 
smaller than 4, u / 2 can be rewritten asu/2»r + s/ 4. When s Is 0 or 2. no rounding is required since u / 2 is already 
an integral multiple of 1 / 2. However when s is equal to 1 or 3, the value of s is rounded to 2. By increasing the possft>illty 

25 that s takes the value of 2 using this rounding method, the filtering effect of motion compensation can be emphasized. 
When the probability that the value of s prior to rounding is 0, 1 , 2, and 3 are ail 25 percent, the probability that 8 will be 
0 or 2 after rounding will respectively be 25 percent and 75 percent. The above explained process related to the hori- 
zontal component u of the motion vector is also applied to the vertical component v. Accordingly, in the U block and V 
block, the probat^iity for using the intensity value of the 401 position is 1 / 4 X 1 / 4 » 1 / 16. and the probabifity for using 

30 the intensity value of the 402 and 403 positions is both 1/4X3/4^3/16, while the probability for using the intensify 
value of position 404 is 3 / 4 X 3 / 4 « 9 / 1 6. By utilizing the same method as above, the expectation for the error of the 
intensity valueisOX 1/16-1^1/4X3/ 16-^1/4X3/ 16-^1/8X9/ 16- 21/128. Just as explained above tor the Y 
block, when interframe encoding is continuously perfonrned, the problem of accumulated errors occurSb 

As related above, for image sequence coding and decoding methods in which interframe prediction is performed 

35 and luminance or chrominance intensity is quantized, the problem of accumulated rounding errors occurs. This round- 
ing error is generated when the luminance or chrorranance intensity value is quantized during the generation of the 
interframe prediction image. 

In view of the at)Ove problems, it is therefore an object of this invention, to improve the quality of the reconstructed 
Image by preventing error accurmjlation. 
40 In order to achieve the alx>ve object, the accumulation of errors is prevented by limiting the occunence of enors or 
performing an operation to cancel out errors that have occurred. 

BRIEF DESCRIPTION OF THE DRAWINGS 

45 . Figure 1 is a block diagram showing the layout of the H.263 image encoder. 
Figure 2 is a block diagram showing the layout of the H.263 image decoder. 
Figure 3 is a drawing showing the structure of the macro block. 

Figure 4 is a drawing showing the interpolatfon process of intensity values for block matching with half pixel accu- 
racy. 

so Rgure 5 is a drawing showing a coded image sequence. 

Figure 6 is a block diagram shewing a software image encoding device. 

Rgure 7 is a block diagram showing a software inoage decoding device. 

Figure 8 is a flew chart showing an example of processing in the software Image encoding device 

Rgure 9 is a f tow chart showing an example of the coding mode decision processing for the software image encod- 
55 ing devica 

Figure 1 0 is a flow chart showing an example of motion estimation and motion oonpensation processing in the soft- 
ware image encoding device. 

Rgure 1 1 is a flow chart showing the processing in the software image decoding device. 
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Figure 12 is a llow chert shoving ar^ example ol motion compensation processing in the software image deoodtng 
dev/ice. 

Figure 13 is a drawing showing an example of a storage media on which an encoded bit stream generated by an 
encoding method that outputs bit streams including 1, and P- frames is recorded. 
5 Figure 14 is a set of drawings showing specific examples of devices using an encoding method where P-f and P- 
f rames coexist 

Figure 15 is a drawing showing an example of a storage media on which an er^oded bit stream generated by en 
encoding method the outputs bit streams including I. B, P-f. and P-frames recorded. 

Figure 16 is a block diagram showing an example of a block matching unit included In a 6w\ce using an encoding 
10 method where P-f ard P- frames coexist. 

Figure 1 7 is a block diagram showing the prediction image synthesizer included in a de^ce for decoding bit streama 
encoded by an encoding method where P-f and P* frames coexist 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

First in which circumstances the accumulated rounding errors as related in the Trior art' occur must be consid- 
ered. An example of an image sequences encoded by coding methods which can perform both unidirectional predictk)n 
and bidirectional prediction such as in MPEG. 1 , MPEG. 2 and H. 263 is shown in Fig. 5. An image 501 is a frame-^xxled 
by means of intraframe coding and is referred to as an I frame. In contrast images 503, 505, 507, 509 are called P 

20 friames and are coded by unidirectional intertrame coding by using the prevk>u& I or P frame as the reference image. 
Accordingly, when for instance encoding image 505. image 503 is used as the reference image arxJ intertrame predic- 
tion is performed. Images 502, 504, 506 and 508 are called B frames and bidirectional intertrame prediction is per- 
formed utilizing the previous artd subsequent I or P frame. The B frame is characterized bf not being used as a 
reference image when interframe prediction is performed. Since nmtion compensation is not performed in I frames, the 

25 rounding error caused by motion compensation will not occur. In contrast, not only is motion compensaticn perlornied 
in the P frames but the P frame is also used as a reference image by other P or B frames so that K may be a cause 
leading to accumulated rounding errors. In the B frames on the other hand, motion compensation is perfbrmed 60 that 
the effect of accumulated rounding errors appears in the reconstructed image. However, due to the fact that B frames 
are not used as reference images, B frames cannot be a source of accumulated rounding errors. Thus, if aocumulated 

30 rounding errors can be prevented in the P frame, then the bad effects of rounding errors can be alleviated In the overall 
image sequence In H.263 a frame for coding a P frame and a B frame exists and Is called a PB frame (For instance. 
franr>es 503 and 504 can both be encoded as a PB frame.) If the confined two frames are viewed as separate framee. 
then the same principle as above can be applied. In other words, if countermeasures are taken versus rounding errors 
for the P frame part within a PB frame, then the accumulation of errors can be prevented. 

35 Rourxling errors occur during interpolation of intensity values when a value obtained from normal division (divisaon 
whose operation result is a real number) is a haK integer (0.5 added to an integer) an6 this result is then rounded up to 
the next integer in the direction away from zero. For instance, when dividing by 4 to find an interpolated intensity value 
is performed, the rounding errors for the cases v^en the residual is 1 and. 3 have equal absolute values but different 
signs. Consequently, the rourxiing errors caused by these two cases are car)celed when the expectation for tfie round- 

40 ing errors is calculated (in more general words, when dividing by a positive integer d* is performed, the rounding errors 
caused by the cases when the residual is t and d* -t are cancelled). However, when the residual is 2, in other words 
when the result of normal division is a half integer, the rounding error cannot be canceled arxJ leads to accumulated 
errors. To solve this prot)lem, a method that allows the usage of two roundir^ methods can be used. The two rounding 
methods used here are: a rounding method that rounds half integers away from 0; and a rourxSng method that rounds 

45 half integers towards 0. By combining the usage of these two rounding methods, the rounding errors can be canceled. 
Hereafter, the rounding method that rounds the result of normal division to the nearest integer and rounds half integer 
values away from 0 is called "positive rounding*. AdditionaHy, the rounding method that rounds the result of rwrmal dh^i- 
. sion to the nearest integer and rounds half integer values towards 0 is called "negative rounding*. The process of poe- 
itive rounding used in block matching with half pixel accuracy is shown in Equation 3. When negative rounding is used 

60 instead, this equation can be rewritten as shown beloML 
[Equation ^ 

/a B La . (4) 

Ic » l{LB*LjC)f2H 
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/d-I(La+lfe*Lc+Ld+l)/4] 

Hereafter rTX)tk>n compensation methods that performs positive and negative rounding for the synthesis of Inter- 
frame prediction images are called * motion compensation using positive rounding' arxl 'nfx>tion compensation using 
6 negative rounding', respectively. Furtheimore. for P frames which use WoCk matching with half pixel accuracy for 
motion compensation, a frame that uses positive rounding is called a *P+ frame "and a frame that uses negative round- 
ing 18 called a 'P- frame ' (under this definition, the P frames in H. 263 are all P+ frames). The expectation for the round- 
ing enors in P+ and P- frames have equal absolute values but different signs. Accordingly, the accumulation of rounding 
errors can be prevented when P+ frames and P- frarries are altemately located along the time axis. In the example In 

10 Fig. 5. if the frames 503 and 507 are set as P+ frames and the frames 605 and 509 are set as P- frames, then this 
method can be implemented. The alternate occurrence of P-r frames and P- frames leads to the usage of a P+ frame 
and a P- frame in the t)idirectional prediction for B frames. Generally, the average of the fonvard prediction image (I.e. 
the prediction image synthesiz ed by using frame 503 when frame 504 In Fig. 5 is being encoded) and the backward pre- 
diction image (i.e. the prediction image synthesized by using frame 505 wrhen frame 504 in Fig. 5 is being encoded) is 

16 frequently used for synthesizing the prediction irriage for B frames. This means thai using a P+ frame (which has a pos- 
itive value for the expectation of the rounding error] and a P- frame (which has a negative value for the expectation of 
the rouncSng enor) in bidirectional prediction tor a B frame is effective in canceling out Ihe effects of rounding erroiB. 
Just as related above, the rounding process in the B frame will not be a cause of error accumulation. Accordingly, no 
problem will occur even H the same rounding method is applied to all the B frames. For instance, no serious degradation 

20 of decoded images is caused ©/en if motion compensation using positive rounding is performed for all of the B frames 
502, 504. 506. and 508 in Fig. 5. Preferably only one type of rouriding is performed for a B frame, in order to simplify 
the B frame decoding process. 

A block matching section 1600 of an image encoder according-to the above described motion compensation 
method utilizing multiple rounding methods is shown in Fig. 16. Nun^ers identfcal to tix)se in other drawings indicate 

25 the same part. By subslhuting the block matching section 1 16 of Rg. 1 with 1600, multiple rounding nnethods can be 
used. Motion estimation processing between the input image 101 and the decoded image of the previous frame is per- 
formed in a motion estimator 1601. As a result, motion information 120 is output. This motion information is utilized in 
the synthesis of the prediction image in a prediction image synthesizer 1603. A roiKxJing method determination device 
1602 determines whether to use positive rounding or negative rounding as the rounding method for the frame cun-enHy 

30 being encoded. Information 1604 relating to the rounding method that was determined is input to the prediction image 
synthesizer 1603. In this prediction image synthesizer 1603. a prediction image 117 is synthesized and output based 
on tiie rounding method determined by means of Information 1604. In the block matching section 116 in Rg. 1, there 
are no items equivalent to 1602. 1604 of Rg. 16, and the predictton image is synthesized only by positive rounding. 
Also, the rounding method 1 605 determined at the block matching section can be output and this tnfornr>ation can then 

35 be multiplexed into tiie bit stream and be transmitted. 

A prediction image synthesizer 1700 of an image decoder which can decode b'rt streams generated t)y a coding 
method using multiple rounding methods is shown in Fig. 17. Numbers identical to those in other drawings indicate tiie 
same part. By sutjstituting the prediction image synthesizer 211 of Fig. 2 by 1700. multiple rounding methods can be 
used. In the rounding method determination device 1701, the rounding mettiod appropriate for predictton image syn- 

40 thesis in the decoding process is determined. In order to cany out decoding correctiy. the rounding method selected 
here must be the same as the rounding method that was selected for encoding. For instance the following rule can be 
shared between the encoder and decoder: When the current frame is a P frame and the number of P frames fincluding 
the current frame) counted from the most recent I frame is odd. then the current frame is a P+ frame. When this nuo^er 
is even, then the current frame is a P- frame. If the rounding method determination device on tiie encocfing side (Fa 

45 instance, 1602 in Rg. 1 6) and the rounding method determination device 1701 conform to this common rule, then the 
images can correctiy be decoded. The prediction image is synthesized in the predictfon image synthesizer 1703 using 
motion information 202. decoding image 210 of the prfor frame, and information 1702 related to the rounding method 
determined as just described. This prediction image 212 is output and then used fv the synthesis of the decoded * 
image. As an altemative to the above mentioned case, a case where the informatk>n related to the rounding method is 

50 multiplexed in the transmitted bit stream can also be considered (such bit stream can be generated at the encoder by 
outputting the information 1605 related to the rounding method from the block matching section depicted in Rg. 16). In 
such case, the rounding method determiner devfoe 1701 is not used, and information 1704 related to the rounding 
method extracted from the encoded bit stream is used at the prediction Image synthesizer 1703: 

Besides the image encoder and the image decoder utilizing the custom circuits and custom chips of the conven- 

55 tional art as shown in Rg. 1 and Fig. 2. this invention can also be applied to softv^/are image encoders and software 
image decoders utilizing general-purpose processors. A software image encoder 600 and a software image decoder 
700 are shown in Rg. 6 and Rg. 7. In the software image encoder 600, an input image 601 is first staed in the ii^ 
frame memory 602 and the general-purpose processor 603 loads information from here and performs encodir^. The 
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program for driving this general-purpose processor is loaded from a storage device 608 which can be a hard difik, flc(3py 
disk, etc. and stored in a program memory 604. This general-purpose processor also uses a process memory 605 to 
perform the encoding. The encoding information output by the general-purpose processor Is temporarily stored In th« 
output buffer 606 and then output as an encoded bit stream 607. 

5 A flowchart for the encoding software {recording medium readable by corrputer) is shown in Rg. 8. The process 
starts in 801 . and the value 0 is assigned to variable N in 802. Next in 803 and 804. the value 0 Is assigned to N when 
the value for N is 100. N is a counter for the number of frames. 1 is added for each one franr^e whose processing is com- 
plete, and values from 0 to 99 are allowed when performing coding. When the value for N is 0, the current frame is an 
I frame. When N is an odd number, the current frame is a P+ frame, and when an even number other than 0, the cun'snt 

10 frame is a P- frame. When the upper limit for the value of N Is 99. it means that one I frame is coded after 99 P frames 
(P+ frames or P- frames) are coded. By always inserting one I frame in a certain number of coded frames, the fonowing 
benefits can be obtained: (a) Error accumulation due to a mismatch between encoder and decoder processing can be 
prevented (for instance, a mismatch in the conputation of OCT); and (b) The processing load for acquinng the repro- 
duced image of the target frame from the coded data (random access) is reduced. The optimal N value varies when the 

15 encoder performance or the environment where the encoder is used are changed. It does not mean, therefore, that the 
value of N must always be 100. The process for determining the rounding method and codirig mode for each frame is 
performed in 805 and the flowchart with details of this operation is shiown in Fig. 9. First of all, whether N is a 0 or not 
is checked in 901 . If N is 0. then T is output as distinction inforniation of the prediction mode, to the output Ixjffer in 902. 
This means that the image to be coded is will be coded as an I frame. Here, "output to the output buffer" means that 

20 after being stored in the output buffer, the information is output to an external device as a portion of the coded bit 
stream. When N is not 0, then whether N is an odd or even nun*)er is identified in 904. When N is an odd nurrtw, V 
is output to the output Ixjffer as the distinction information for the rounding method in 905. and the image to be coded 
will be coded as a P+ frame. On the other hand, when N is an even number, is output to the output buffer as the dis- 
tinction information for the rounding method in 906. and the innage to be coded will be coded as a P- frame. The process 

ss again returns to Fig. 8, where after determining the coding mode in 805, the input image is stored in the frame memory 
A in 806. The frame memory A referred to here signifies a portion of the memory zone (for instance, the memory zone 
maintained in the memory of 605 in Fig. 6) of the software encoder. In 807, it is checked whether the frame currently 
being coded is an I frame. When not identified as an I frame, motion estimation and motion conpensation is performed 
in 808. The flowchart in Fig. 10 shows details of this process performed in 808. First of all, in 1001. nrwtion e6timatk)n 

30 is performed between the images stored in frame mentories A and B (just as written in the final part of this paragraph, 
the decoded image of the prior frame is stored in frame memory B). The motton vector for each block is found, and this 
motion vector is sent to the output buffer. Next in 1002, whether or not the cunent frame is a P+ frame is checked. 
When the cunent frame is a P+ frame, the prediction image is synthesized in 1003 utilizing positive rounding and this 
prediction image is stored in frame memory C. On the other hand, when the current frame is a P- frame, the predictten 

S5 image is synthesized in 1004 utilizing negative rounding and this prediction image is stored in the frame memory C. 
Next, in 1005, the differential image between frame memories A and C is found and stored in frame memory A. Here, 
the process again returns to Rg. 8. Prior to starting the processing in 809. the input image is stored in frame memory 
A when the current frame is an I frame, and the differential image between the irput image and the prediction image is 
stored in frame merrory A when the current frame is a P frame {P+ or P- frame).. In 809, DCT is applied to the image 

40 stored in frame memory A, and the DCT coefficients calculated here are sent to the output buffer after being quantized. 
In 810, inverse quantization is performed to the quantized DCT coefficients and inverse DCT is applied. The image 
obtained by applying inverse DCT is stored in frame merTx>ry B. Next in 811, it is checked again wrhether the cunent 
frame is an I frame. When the current frame is not an 1 frame, the images stored in frame memory B and 0 are added 
and the resuH is stored in frame memory a The coding process of a frame ends here, and the image stored in frame 

4S memory B before going into 81 3 is the reconstructed image of this frame (this image is identical with the one obtained 
at the decoding side). In 813. it is checked whether the frame whose coding has just finished is the final frame in the 
sequence. If this is true, the coding process ends . if this frame is not the final frame. 1 is added to N In 814, and the 
process again returns to 803 and the coding process for the next frame starts. 

A software decoder 700 is shown in Fig. 7. After the coded bft stream 701 is temporanTy stored in the Input buffer 

50 702, this bit stream is then loaded into the general-purpose processor 703. The program for driving tWs generaliJur- 
pose processor is k>aded from a storage device 708 vitiich can be a hard disk, floppy disK etc. and stored in a program 
memory 704. This general-purpose processor also uses a process memory 605 to perform the decoding. The decoded 
image obtained by the decoding process is temporarily stored in the output frame memory 706 and then sent out as the 
output image 707. 

« A flowchart of the decoding software for the software decoder 700 shown in Fig. 7 is shown In Fig. 1 1 . The process 
starts in 1 1 01 . and ft Is checked in 1 1 02 v/hether input information is present If there is no input information, the decod- 
ing process ends in 1 103. When input information is present, distinction Information of the prediction mode is irpul in 
1104. The word InpuT used here means that the information stored in the input buffer (fbr instance 702 of Fig. 7) is 
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lofided by the general-purpose proceesor. In 1 105. rt is checked whether the encoding mode distinction intorrnation Is 
"1". When not "I", the distinction information for the romding method Is input and synthesis of the Interlran^e precfiction 
image Is pertormed In 1 107. A flowchart showing details o! the operation In 1 107 Is shown In Fig. 12. In 1201, a motion 
vector is input lor each block. Then, in 1202. it is checked whether the distinction information tor the rounding method 

5 loaded in 1 1 06 Is a When this information is the frame currently being decoded Is a P4 frame. In this case, the 
prediction image is synthesized using positive rounding in 1203. and the prediction image is stored in frame memory D. 
Here, frame memory 0 signifies a portion of the memory zone of the software decoder (fa instance, this memory lone 
is obtained In the processing memory 705 in Fig. 7). When the distinction information of the rounding method i$ not V, 
the current frame being decoded is a P- frame. The prediction image is synthesized using negative rounding In 1204 

10 and this prediction image is stored in frame menx)ry D. At this point If a P-t- frame Is decoded as a P- frame due to some 
type of error, or conversely it a P- frame is decoded as a P+ frame, the correct predictiori Image Is not synthesized In 
the decoder and the quality of the decoded image deteriorates. After synthesizing the prediction image, the operation 
returns to Fig. 11 and the quantized DCT coefficients is input in 1108. Inverse quantization and Inverse OCT Is then 
applied to these coeff icients and the resulting image is stored in frame memory E. In 1 109. H is checked again whether 

IS the frame currently being decoded is an I frame. If the current frame is not an I frame, images stored in frame memory 
D and E are added in 1 1 10 and the resulting sum image Is stored in frame memory E. The image stored In frame mem- 
ory E before starting the process in 1 1 1 1 is the reconstructed image. This image stored in frame memory E is output to 
the outpul frame memory (for instance. 706 in Fig. 7) in 1 1 1 1 . and then output from the decoder as the reconstructed 
image. The decoding process for a frame is conpleted here and the process for the next frame starts by returning to 

20 1102. 

When a software based on the ftowchart shown in Figs. 8 - 12 is run in the software image encoders or decoders, 
the same effect as when custom circuits and custom chips are utilized are obtained. 

A storage media (recording media) with the bit stream generated by the software encoder 601 of Fig. 6 being 
recorded is shown in Fig. 13. It is assumed that the algorithms shown in the flowcharts of Figs. 8 • 10 is used in the 

25 software encoder. Digital information is recorded concentrically on a recording disk 1301 capable of recording digital 
information (for instance magnetic disks, optical disk. etc.). A portion 1302 of the information recorded on tWs digital 
disk includes: prediction mode distinction information 1303, 1305. 1308, 1311. and 1314; rounding nr>ethod distinction 
information 1306, 1309, 1312, and 1315; and motion vector and DCT coefficient information 1304, 1307, 1310. 1313, 
and 1316. Information representing T is recorded in 1303. V is recorded in 1305, 1308, 1311. and 1314, V is recorded 

30 in 1 306. and 1312. and is recorded in 1309, and 131 5. In this case, T and V can be represented by a single bit of 0, 
and 'P and can be represented by a single bit of 1 . Using this representation, the decoder can conectly interpret the 
recorded information and the correct reconstructed image is synthesized. By storing a coded bit stream in a storage 
media using the method described above, the accumulation of rounding errors is prevented when the bit stream Is read 
and decoded. 

35 A storage mecfia wrth the bit stream of the coded data of the image sequence shown in Fig. 5 being recorded is 
shown in Fig. 15. The recorded bit stream includes information related to P+. P-. and B frames. In the same way as in 
1301 of Fig. 13. digital information is recorded concentrically on a record disk 1501 capable for recording digital infcx- 
mation(for instance, magnetic disks, optical disks, etc.). A portion 1502 of the digital infornnation recorded on this digitaJ 
disk includes: prediction mode distinction information 1503. 1505. 1508. 1510. and 1513; rounding method distinction 

40 information 1506. and 1512; and motion vector and DCT coefficient information 1504, 1507. 1509. 1511. and 1514. 
Information representing T is recorded in 1503. P' is recorded in 1505, and 1510. 'B' is recorded in 1508. and 1513. V 
is recorded In 1505. and is recorded in 1511. In this case. T, P' and 'B' can be represented respectively by two bit 
values 00, 01 , and 10, and V and can be represented respectively by one bit values 0 and 1 . Using this representa- 
tion, the decoder can correctly interpret the recorded information and the correct reconstructed is synthesized. In Rg. 

45 15, information related to frame 501 (I frame) In Fig. 5 is 1503 and 1504, information related to 502 (B frame) is 1508 
and 1 509. information related to frame 503 (P+ frame) is 1 505 and 1 507, information related to frame 504 (B frame) is 
1513 and 1 514. and information related to frame 505 (P- frame) is 1510 and 1512. When coding image sequences m 
coded using B frames, the transmission order and display order of frames are usually different. This is because the pre- . 
vious and subsequent reference images need to be coded before the prediction image for the B franr>e is synthesized. 

so Consequently, in spite of the fact that the frame 502 is displayed before frame 503, information related to frame 503 is 
transmitted before information related to frame 502. As described above, there is no need to use muH^e rounding 
methods for B frames since motion compensation In B frames do not cause accumulation of rounding errors. Therefore, 
as shown in this otarrple. informatbn that specifies rounding methods (e.g. and -*) is not transmitted for B frames. 
Thus for instance, even if only positive rounding is applied to B frames, the problem of accumulated rounding errors 

S5 does not occur. By storing coded bit streams containing Information related to B frames in a storage media in the way 
described above, the occurrence of accumulated rounding en^ors can be presented wrtien this bit stream is read and 
decoded. 

Specific examples of coders and decoders using the coding method descn^bed in this specification is shown in Rg. 
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14. The image coding and decoding method can be utilized by installing image coding and decoding software into a 
computer 1401 . This software is recorded in some kind of storage media (CD-ROM, floppy disk, hard disk, olc) 1412. 
loaded into a corrputer and then used. Additionally, the computer can be used as an Image communication terminal t>y 
conneding the cornpirter to a communication lines. H is also possible to install the decoding method desaibed in tWs 
specification into a player device 1 403 that reads and decodes the coded bit stream recorded In a storage mecSa 1402. 
In this case, the reconstructed image signal can be displayed on a television monitor 1404. The device 1403 can be 
used only for reading the coded bit stream, and in this case, the decoding device can be installed in the television mon- 
itor 1404. It is well known that digital data transmission can be realized using satellrtes and terrestrial waves. A decoding 
device can also be installed in a television receiver 1 405 capable of receiving such digital transmissions. Also, a decod- 
ing device can also be installed inside a set top box 1409 connected to a satellite/tenrestrial wave antenna, or a cable 
1408 of a cable television system, so that the reconstructed images can be displayed on a television monitor 1410. In 
this case, the decoding device can be incorporated in the television monitor rather than in the set top Ixw, as in the case 
of 1404. The layout of a digital satellite broadcast system is shown In 1413. 1414 and 1415. The video Information In 
the coded bit stream is transmitted from a broadcast station 1413 to a communication or broadcast satellite 1414. The 
satellite receives this information, sends it to a home 1415 having equipment for receiving satellite broadcast programs, 
and the video information is reconstructed and displayed in this home using devices such as a television receiver or a 
set top box. Digital image communication using mot>le terminals 1406 has recently attracted considerable attention, 
due to the fact that image communication at very low bit rates has become possik)le. Digital portable terminals can be 
categorized in the following three types: a transceiver having both an encoder and decoder; a transn^er having only 
an encoder; and a receiver having only a decoder. An encoding device can be installed in a video camera recorder 
1407. The camera can also be used just for capturing the video signal and this signal can be supplied to a custom 
encoder 1411. AH of the devices or systems shown In this drawing can be equipped with the coding or/and decoding 
method described in this specification. By using this coding or/and decoding method in these devices or systems, 
images of higher quality compared with those obtained using conventional technologies can be obtained 
The following variations are clearly included within the scope of this inventioru 

(i) A prerequisite of the above described principle was the use of block matching as a motion compensatXNi 
method. However, this invention is further capable of being applied to all image sequence coding and decoding 
methods in which motion compensation is performed by taking a value for the vertical and horizontal components 
of the pixel motion vector that is other than an integer multiple of the sampling period in the vertical and horizorital 
directions of the pixel, and then finding by interpolation, the intensity value of a position where the sample value is 
not present. Thus for instance, the global motion compensation listed in Japanese Patent Application No. Hei 08- 
060572 and the warping prediction listed in Japanese Patent Application No, Hei 08-249601 are applicable to the 
method of this invention. 

(ii) The description of the invention only mentioned the case where a value integral multiple of 1/2 was taken for the 
horizontal and vertical components of the motion vector. However, this invention is also generally applicable to 
methods in which Integral muhiples of l/d (d is a positive integer and also an even nunnber) are allowed for the hor- 
izontal and vertical components of the motion vector. However, when d becomes large, the divisor for division in 
bilinear interpolation (square of d. see Equation 2) also becomes large, so that In contrast, the probability of results 
from normal division reaching a value of 0.5 become low. Accordingly, when performing only positive rounding, the 
at)solute value of the expectation fbr rounding errors becomes small and the bad effects caused tsy aocumulated 
errors become less conspicuous. Also applicable to the method of Hvs inventioa is a motion compensation method 
where fbr instance, the d value is variable, both positive rounding and negative rounding are used when d is smaBer 
than a fixed value, and only positive rounding or only negative rouncfing is used when the value of d Is larger than 
a fixed valua 

(iii) As mentioned in the prior art when DCT Is utilized as an error coding method, the adverse effects from accu- 
mulated rounding errors are prone to appear when the quantized step size of the DCT coefficient is large. However 
a method is also applicable to the invention, in which, when the quantization step size of DCT coefficients is larger 
than a threshold value then both positive rounding and negative rounding are used. When the quantization step 
size of the DCT coefficients is smaller than the threshold value then only positive rounding or only negative round- 
ing is used. 

(iv) In cases where error accumulations occur on the luminance plane and cases where error accumulations occur 
on the chrominance plane, the bad effects on the reconstructed images are g^erally more serious in the case of 
error accumulations on tiie chrominance plane. This is due to the fact that rather than cases where the image darfc- 
ens or Tightens slightly, cases where overall changes in tiie image cotor happen are more conspicuous. Hcwever. 
a method is also applicable to this invention in which both positive rounding and negative rounding are used lor the 
chronunance signal, and only positive rounding or negative rounding is used for the luminance signal. 

As described in the description of related art 1/4 pixel accuracy motion vectors obtained by halving the 1/2 
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pixel accuracy motion vectors are rourxjed to 1/2 pixel accuracy in H.263. Hov^ever by adding certain changes to 
this method, the absolute expectation value tor rounding errors can be reduced. In H. 263 that was mentioned in 
the prior art. a value which is half the horizontal or vertical components of the motion vector lor the luminance plane 
is expressed as r 4- s / 4 (r is an integer, s is an integer less than 4 and not smaller than 0), and when s is 1 or 3, a 
rounding operation is performed to obtain a 2. This operation can be changed as follows: When s Is 1, a rounding 
operation is performed to obtain a 0, end when s is 3 a 1 is be added to r to make s a 0. By performing these oper* 
ations. the number of times that the intensity values at positions 406 - 408 in Fig. 4 Is definitely reduced (Probatrfilty 
that horizontal and vertical components of motion vector will be an integer becon^e high.) so that the absolute 
expectation value for the rounding error becomes small. However, even if the size of the error occumng in this 
method can be limited, the accumulation of errors cannot be completely prevented. 

(v) The invention descrit>ed in this specification is applicable to a method that obtains the final interframe prediction 
image by averaging the prediction images obtained by different motion compensation methods. For exanrple. in the 
method described in Japanese Patent Application Na Hei 8-2616, interframe prediction inriages obtained by the fol- 
lowing two methods are averaged: block matching in which a motion vector is assigned to each 16x16 pixel block: 
and block matching in which a motion vector is assigned to each 8x8 pixel t3locks. In this method, rounding is also 
performed when calculating the average of the two prediction images. When only positive rounding is continuously 
performed in this averaging operation, a new type of rounding error accumulates. This problem can be solved by 
using multiple rourxiing methods for this averaging operation. In this method, negative rounding is performed in the 
averaging operation when positive rounding is performed in block matching. Conversely, positive rounding is used 
for the averaging when negative rounding is used for Uock matching. By using different rounding methods for aver- 
aging and block matching, the rounding errors from two different sources is cancelled within the sanne framed.. 

(vi) When utilizing a method that alternately locates P+ frames and P- frames along the time axis, the encoder or 
the decoder needs to determine wrtiether the currently processed P frame is a P+ frame or a P- frame. The following 
is an example of such identification method: A counter counts the number of P frames after the most recently coded 
or decoded I frame, and the current P frame is a P-i- frame when the number is odd, and a P- frame when the 
number is even (this method ts referred to as an implicit scheme). There is also a method for instance, that writes 
Into the header section of the coded image information, information to identify whether the currently coded P frame 
at the encoder is a P^ frame or a P- frame (this method is referred to as an explicit scheme).. Conpared with the 
implicit method, this method is well at)le to withstand transmission errors., sir^ce there is no need to count the 
number of P framea. 

Additionally, the explicit method has the followirig advantages: As described in "Description for Related Arf . 
past encoding standards (such as MPEG-1 or MPEG^2) use only positive rounding for nwtion compensation. This 
means for instance that the motion estimation/motion compensation devices (for exanrple equivalent to 106 In 
Fig.l) for fs/lPEG-1/MPEG-2 on the market are not corrpatiWe with coding methods that uses both P+ frames and 
P> frames. It is assumed that there is a decoder which can decode bit streams generated by a coding method that 
uses P-t^ frames and P- frames. In this case rf the decoder is based on the above mentioned implicit method, then 
it will be difficult to develop an encoder that generates bit streams tiiat can tie correctly decoded by the above men* 
tioned decoder, using the above mentioned motion estimation/compensation device for MPEG-1/MPEQ-2. How- 
ever, rf the decoder is based on the above mentioned explicit method, this problem can be solved. An encoder using 
an MPEG'l/r\^PEGT2 motion estimation/motion compensation device can continuously send P4 frames, by contin- 
uously writing rounding method distinction infanr^tion indicating positive rounding into the frame information 
header. When this is performed, a decoder based on the explicit method can conectly decode the txt stream gen- 
erated by this encoder. Of course, it should be nrx>re likely in such case that the accumulation of rourxJing errors 
occurs, since only F-t- frames are present However, error accumulation is not a serious problem in cases where the 
encoder uses only small values as the quantization step size for the DOT coefficients (an example for such ooders 
is a custom encoder used only for high rate coding). In addition to this interoperabTity between past standards, the 
explicit method further have the foltowing advantages: (a) the equipment cost tor high rate custom encoders and 
coders not prone to rounding error accumulation due to frequent insertion of I frames can be reduced by InstalGng 
only positive or negative rounding as the pixel value rounding method for motion compensation; and(b) the above 
. encoders not prone to rounding error accumulation have the advantage in that there is no need to decide whether 
to code the current frame as a P-f or P- frame, and the processing is simplified. 

(vii) The invention described in this spedfication is applicable to coding and decoding methods ttiat applies fflt^ing 
accompanying roundir^ to the interframe prediction images. For instance, in the international starxJard H. 261 for. 
image sequence coding, a bw-pass filter (called a loop filter) is applied to block signals whose motion vectors are 
not 0 in interframe prediction images. Also, in H. 263, filters can be used to smooth out discontinuities on bfock 
boundaries (blocking artifacts). Ail of these filters perform weighted averaging to pixel intensity values and rounding 
IS then performed on the averaged intensity values. Even for these cases, selective use of positive rourxfing and 
negative rounding is effective for preventing ena accumulation. 
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(viii) Besides I P4 P- P+ P- .... various methods lor mixing P+ frames and P- frames such as I P+ P+ P- P- P+ P+ 
.... or 1 P4 P- P- P-f P4 ... are applicable to the method of this, invention. For instance, using a random number gen- 
erator that outputs 0 ard 1 both at a probability of 50 percent, the encoder can code a P-f and P- frame when the 
output is 0 and 1 . respectively. In any case, ttie less the difference in probability that P-f frames and P- frames occur 
in a certain period of time, the less the rounding error accumulation Is prone to occur. Further, when the encoder Is 
allowed to mix P4 frames and P- ftamee by an arbitrary method, the encoder and decoder must operate based on 
the explicit method arxj rK)t with the implicit method described above. Accordingly, the explicit method is superior 
when viewed from the perspective of allowing f lextoiltty configuration for the encoder and decoder. 

(ix) The invention described in this specification does not limit the pixel value interpolation method to bilinear Inter- 
polation. Interpolation methods for intensity values can generally be described by the following equation: 

[Equations] 

jr X 

R(x*r.y*s)'T('£'£h{r-j.s-k)R(x*j.y*k)) (5) 

where, r and s are real numbers. h(r. s) is a function for interpolating the real numbers, and T(z) is a function 
for rounding the real number 2. The definitions of R (x, y), x. and y are the same as in Equation 4. faction compen- 
sation utilizing positive rounding is performed when T (z) is a function representing positive rounding, and motion 
compensation utilizing negative rounding is performed when the function representing negative rounding. This 
invention is applicable to interpolation methods that can be described using Equation 5. For instance, bilirtear inter- 
polation can be descrit)ed by defining h(r. s) as shown below. 

[Equation 6] 

A(r.5) = (l-|r|)(U|5|). 0s|r|^l,0s|5|sl. .-.(6) 
0, otherwise. 



However, If for instance h(r,8) is defined as shown below, 
[Equation 7] 

Hf^^s) = 1 - H - 14 0 s \r\ + |j| s 1, rj < 0, 

1-H. |r|^|5|, |r| ^l,r5aO. -(7) 

|.v|>H, |5|sl,r5>0, 
0, otherwise. 



then an Interpolation metiiod different from bilinear interpolatiGn Is implentented but the invention is stOl 
applicable. 

(x) The invention desaibed in this specification does not timft the coding method for error images to OCT (disaete 
cosine transform). For Instance, wavelet transform (tor exanrple, M. Antonioni, et al. 'Image Coding Using Wavelet 
Transform" IEEE Trans. Image Processing, vol. 1, no2, April 1992) and Walsh-Hadamard transform (for example, 
A. N. Netravalli and 6. G. Haskell, "Digital Pictures*, Plenum Press, 1998) are also applicable to this invention. 

Claims 

1 . An Image sequence coding method conprising the steps of: 

synthesising a prediction image by utilising motion compensation, and 

multiplexing information related to the difference image between an input image and said prediction image, and 
information related to motion vectors estimated during said motion compensation, 

wherein there is a case wfiere said prediction image is synthesised by motion compensation using pos- 
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Hive rounding, and a ca&e where eaid prediction image is Bynthesised by motion compensation using negative 
rounding. 

2. The method of claim 1 . wher ein infor malion related to the rounding method used for the synthesle of said prediction 
image is multiplexed with said information related to said difference image arxi said information related to motion 
vectors. 

3. An in^ge sequence coding method comprising the steps of: 

synthesising a prediction image by utilising nx)tion compensation. arKf 

multiplexing information related to the difference image between an input inrage arxi said prediction image, and 
information related to motion vectors estimated during said motion compensation. 

wherein information related to the rounding method used for the synthesis of said prediction image is 
multiplexed with said information related to said difference image and said information related to motion vec- 
tors. 

4. An image sequence coding method comprising the steps of: 

synthesising a prediction image by utilising motion compensation between an input image and a reference 
image, and 

multiplexing information related to the difference image between said input image arxf said prediction image, 
and information related to motion vectors estimated during said nx)tion compensation, wherein: 
said motion compensation has half pixel accuracy arvj the intensity values of chrominance or luminance at a 
point on the reference image where no pixels are present are calculated by (bilinear interpolation, and 
there is a case where said bilinear interpolation is calculated by positive rounding according to 
Lb c [(La-i-UHl) /2] . Ic « [(La+Lc-^l)/2] . and Id - [(La-fLbflx-^Ld+2)M] . and a case where said bilinear inter- 
polation is calculated by negative rounding according to lb ^ [(La+Ubyz] , Ic = [(La+Lc)/ 2] , arxl 
Id K ((La+liHLc-f Ld4l)/4] . where La, Lb, Lc. and Ld are respectively the intensity values of a first pixel, a sec- 
ond pixel which is horizontally adjacent to said first pixel, a third pixel which is vertically adjacent to said first 
pixel, and a forth pixel which is vertically adjacent to said second pixel and horizontally adjacent to said tNrd 
pixel, and Lb. lc. and Id are respectively the interpolated intensity values of the middle point between said first 
and second pixel, the middle point between said first and third pixel, and the middle point between said first, 
second, third, and fourth pixel. 

5. An image sequence coding method for encoding each frame of an inrtage sequence consisting of plurality of 
frames, comprising the steps of: ^ 

synthesising the prediction image of a current frame from the decoded image of a previously encoded frame 
• and the input image of said current frame by means of a first nx}tion compensation. 

generating information related to the difference image of said predictbn image and the input image of said cur- 
rent frame, 

multiplexing and then outputting information related to said difference image and information related to motion 
vectors estimated by eaid first motion compensation, 

synthesising the decoded image of said current frame by utilising infbnmation related to said difference image 
and prediction image of said current frame, and 

synthesising a prediction image of a future frame by means of a second motion compensation from the 
decoded image of said current frame and the input image of said future frame, wherein: 
said first motion compensation and said second motion compensation utilises either positive or negative 
rourxfing for pixel value interpolation, and 

the rounding methods used in said first and second motion compensations are different 

6. The method of clairn 5 wherein information related to the rounding method utilised in said first motion compensation 
is multiplexed with information related to said difference image and information related to said motion vectors. 



7. An image sequence coding method for encoding each frame of an image sequence consisting of a plurality of 
frames by utilising motion conpensation, wherein: 

said image sequence contains a plurality of P frames which perfbrms unidirectional prediction in motion com- 
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pensatrpn. and 

P frames using positive and negative rounding for synthe&ising prediction images appear atternalely along the 
time axis. 

8. An image sequence coding method for encoding each frame of an Image sequence consisting of a pturalrty of 
frames by utilising motion compensation, wherein: 

said image sequence includes an 1 frame arid a plurality of P frames, and different rourxling methods are used 
for odd arxi even 

10 numbered P frames occurring along the time axis from the most recently encoded I frame. 

9. An image sequence coding method for encoding each frame of an image sequence consisting of a plurality of 
frames by utilising motion compensation, wherein a determination whether to use posHive a negative rounding for 
motion compensation of the encoded frame is performed for each frame. 

IB 

10. The method of claim 9. wherein said determination is made based on a random number generator which outputs 
numbers corresponding to each of said posKrve and negative rounding by a possitxirty close to 50 percent. 

11. An image sequence encoder comprising: 

so 

a DCT converter for performing OCT conversion to the difference image between the input image of a current 
frame and the prediction image of said current frame synthesised by motion compensation, 
a quantiser for quantising the converted DCT coefficients, 
a frame memory for storing the decoded image of a reference frame. 
25 a block matching section for estimating motion vectors arxj synthesising the prediction image of said current 

frame by performing motion compensation between the decoded image of said reference frame and the input 
image of said current frame, and 

a multiplexer for multiplexing information related to said quantised DCT coefficients, infornriation related to said 
motion vectors, and information related to the rounding method used for pixel value interpolation in motion 
30 compensation. 

12. An image sequence encoder comprising: 

a DCT converter for performing DCT conversion to the difference image between the input image of a current 
35 frame and the prediction image of said current frame synthesised by motion compensation, 

a quantiser for quantising the converted DCT coefficients, 
a frame memory for storing the decoded image of a reference frame. 

a block matching section for estin^ling nxition vectors and synthesising the prediction inr^ge of said current 
frame by performing motion corrpensation between the decoded image of said reference frame arxf the input 
40 image of said current frame, and 

a multiplexer for multiplexing Information related to said quantised DCT coefficients and information related to 
said motion vectors, wherein: 

said t)lock matching section coirprises a motion estimator for estimating motion vectors, a prediction image 
synthesiser for synthesising prediction images, a rounding method deternrvhation device for deternining 
45 whether the rounding method used for motion compensation is positive or negative rounding and outputting the 

information related to the rounding method determined to use. and 

said prediction image synthesiser synthesises said prediction image based on the said information output from 
said rounding method determination device and information related to said motion vectors output from sidd 
motion estimator. 

60 

13. The encoder of dalm 12. wherein: 

said block matching section outputs said Information related to the rourxfing method used for motion compen- 
sation to said multiplexer, and 

65 said multiplexer multiplexes said information related to the rounding method with said information related to 

said quantised DCT coeffidems and said information related to said motion vectors. 

14. The encoder of claim 12. wherein: 
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said prediction image eynthesiser Eynthesises prediction images of a plurality of P trances, arkd 

different rounding methods ere used for odd Bnd es/en numbered P frames occurring along the time axis from 

the most recently encoded I frame. 

15. An image sequence decoding method corrprising the steps of: 

extracting information related to motion vectors erxi information related to quantised DCT coefficients from 
input information of a decoder, 

synthesising a prediction image utilising motion compensation from said motion vectors and the decoded 
image of a frame decoded in the past, and 

synthesising a decoded image by adding said prediction image to an error image obtained by applying dequan- 
tisation and inverse DCT conversion to said quantised DCT coefficients. 

wherein there are cases vy^hen said prediction image Is synthesised by motion compensation utilisinQ 
positive and negative rounding. 

16. An image sequence decoding method comprising the steps of: 

extracting information related to motion vectors arxl information related to quantised DCT coefiictents from 
Input information of the decoder. 

synthesising a prediction image utilizing motion compensation from said motion vectors arxj a refererKe Image 
v^^hich is the decoded image of a frame previously decoded, and 

synthesising a decoded image by adding said prediction image to an error iimge obtained by applying dequan- 
tisation and inverse DCT conversion to said quantised DCT coefficients, wherein: 

said motion compensation has half pixel accuracy and the intensity values of chrominance or luminance at a 
point on the reference image where no pixels are present are calculated by bilinear interpolation, and 
there is a case where said bilinear interpolation is calculated by positive rounc£ng according to 
lb = [(U-»>LJ>f 1) /2] . Ic = |(La'fLc+1)/2] . and Id = [(La^Lb+U<f|jd-f2)/4] , and a case where said bilinear inter- 
polation is calculated by negative rounding according to lb > [(La-»-Lb)/2!] , Ic « ((La-»-Lc)/ 2] , and 
Id « [(La+l^Lc-^ljd+l)/4] . where La. Lb, Lc. and Ljd are respectively the intensity values of a first pixel, a sec* 
ond pixel which is horizontally adjacent to said first pixel, a third pixel which is vertically adjacent to said first 
pixel, and a forth pixel which is vertically adjacent to said secorxj pixel and horizontally adjacent to said tNid 
pixel, and lb. Ic, and Id are respectively the interpolated intensity values of the middle point between said first 
and second pixel, the middle point between said first and third pixel, and the rrwJdle point between said first 
second, third, and fourth pbcel. 

17. The method of daim 15 or 16. wherein Information specifying either positive or negative rounding is extracted from 
said input information, and the rounding method specified by said information is used in said motion compensation. 

18. An image sequence decoding method for decoding ea^ frame of an image sequence consisting of a plurality of . 
frames, conprising the steps of: 

extracting information related to nrK)tion vectors and quantised DCT coefficients of a first frame from input infor- 
mation, 

synthesising a prediction image of said first frame utilising a first nfX)tion comperisation from the decoded image 
of a previously decoded frame and said motion vectors of said first frame. 

synthesising a decoded image of said first frame by adding said prediction image of said first frame to an error 
image obtained by applying dequantisation and inverse DCT to said quantised DCT coefficients of said first 
frame. 

extracting information related to motion vectors and quantised DCT coefficients of a second frame from said 
input information, and 

synthesising a prediction image of a second utilising a second motion compensation from said decoded image 
of said first frame and said motion vectors of said second frame, wherein: 

said first motion oonpensation and said second nrwtion compensation utilises either positive or negative 
rounding for pixel value interpolation, and 

the rounding methods used in said first motion condensation and said second motion compensation are dif- 
ferent 

19. The method of claim 18. vi/herein information specifying either positive or negative rounding is extracted from said 



14 



EP 0 884 912 A2 

input information, and the rourxjing method &|>ecHied by said information is u&ed in &aid first motion corrpensation. 

20. An image sequence decoding method tor decoding each frame of an image sequence consisting of a plurality of 
fran>es by utilising motion compensation, wherein: ■ 

said image sequence contains a plurality of P frames which performs unidirectional prediction in motion com- 
pensation, and 

P frames using positive and negative rounding for synthestsing prediction images appear alternately along the 
time axis. 

21. An image sequence decoding method for decoding each frame of an image sequence consisting of a plurality of 
frames by utilising motion compensation, wherein: 

said image sequence includes an I frame and a plurality of P frames, and 

different rounding methods are used for odd and even numbered P frames occurring along the time axis from 
the nrx)6t recently decoded 1 frame. 

22. An image sequence decoder comprising: 

a demultiplexer for extracting information related to rTx>tion vectors arxJ information related to quantised DCT 
coefficients from input information, 

a dequantiser for dequantising said quamised DCT coefficients and obtaining DCT coefficients, 
an inverse DCT converter to perform inverse DCT conversion to said DCT coefficient and output an error 
image. 

25 a prediction image synthesiser for syntheslsing a prediction image by motion conpensation using a reference 

image, which is a previously decoded image, and said morition vectors, and 
an adder for adding said error inrtage to said prediction image and output a decoded image. 

wherein there are cases where said prediction image synthesiser uses positive and negative rounding 
for said rnoHon conpensation. 

so 

23. The device of daim 22 wherein: 

said demultiplexer additionally extracts information specifying either positive or negative rounding from said 
input information, and 

35 said prediction image synthesiser uses the rourtding method specified by said information for said motion com- 

pensation. 

24. An image sequence decoder comprising: 

40 a demultiplexer for extracting information related to motion vectors and information related to quantised DCT 

coefficients from input information. 

a dequarrtrser for dequantisir)g said quantised DCT coefficients arxJ obtaining DCT coefficients. 

an inverse DCT converter to perform inverse DCT conversion to said DCT coefficients arxl output an error 

image. 

45 a prediction image synthesiser for synthesising a predction inr^ge by motion compensation using a reference 

image, which is a previously decoded image, and said motion vectors, and 
an adder for adding said en^or image to said prediction image and output a decoded image. 

wherein said prediction image synthesiser has a rourxfing method determination device to determine 
whether to use or negative rounding for said motion compensatiorL 

50 

25. The device of daim 24. wherein: 

an I frame and a plurality of P frames are decoded, and 

different rounding methods are used for odd and even numbered P frames occurring along the time axis from 
55 the most recently decoded I frame. 

26. A computer readable recording medium on which there is recorded an image sequence encoding method compris- 
ing the steps at 
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synlhesising a prediction inwge by utilising nxitibn corrpen^ 

nrtuhiptexing information related to the difference image between an input iniage and said prediction image, and 
information related to motion vectors estimated during said nrx>tion compensation, 

wherein information related to the rounding method used tor the synthesis of said prediction image Is 
5 multiplexed with said information related to said difference image and said infornnation related to motion vec* 

toi*- 

27. A computer readable recording medium on which there is recorded an image sequence decoding method oompris- 
ing the steps d: 

extracting information related to motion vectors, information related to (^entised DOT cbefficients. and infor- 
mation specifying either positive or negative rounding, from input information, 

synthesising a prediction image by motion compensation using said nx>tion vectors and the decoded image of 
a frame decoded in the past, and 
16 a step of synthesising a decoded image by adding said prediction image to an error image obtained by apply- 

ing dequanti&ation and inverse DOT conversion to said quantised DCT coefficients, 

wherein the rounding method specified by said specifying information is used In said motion oompensa* 

lion. 

20 28. A recording medium on which information related to encoded data of an Image sequence is recorded, ¥vherein: 

said infornnation related to encoded data of an image sequence is a set of encoded frame information, and 
said frame information includes information related to the difference image between the input image of the cur- 
rent frame and a prediction image of said current frame syntheeised by means of motion compensation, infor- 
ms mation related to niotion vectors estimated t>y means of said motion compensation, and information for 
distinguishing whether said motion compensation is nrx)tion compensation using positive or negative rounding. 

29. An image sequence coding method utilising a motion compensation method assuming norvnegative integer values 
for chrominance and luminance intensity values allowing the vertical arxi horizontal components of the pixel motion 

30 vectors to take values that are not integral multiples of the pixel sampling period in the vertical and horizontal direc* 
tions, and calculating the Intensity value for a point within a reference image where no pixels are present by means 
of interpolation of sampling values from neightx)uring pixels wherein both positive arKi negative rourxfing are used 
in pixel value interpolation in motion comperisation, where positive rourxjlng rounds Ihel results of bilinear iriterpo* 
lation with real nuniber operations to the nearest integer and rounds half integer values (0.5 added to an integer) 

as awB^ from zera &nd negative rounding differs from positive rounding in that half integer values are rounded towards 
zera 

30. The method of claim 29, wherein motion compensation using either positive or negative rounding is performed tor 
the luminance signal, and motion compensation using both positive and negative rounding is performed tor the. 

40 chrominance signal. 

31. The method of daim 29 or .30 having frames called I frames coded by intraframe coding, P frames coded by Inter- 
frame coding utilising unidirectional prediction for motion conpensation, and B frames coded by irrterframe coding 
utilising bi-directional prediction for motion compensation, wherein P frames to which motion compensation using 

45 positive and negative rounding are applied to luminance and chrominance signals are located alternately along the 
time axis. 

« 

32. The method of daim 29, wherein: 

50 DCT is used for error codirtg. and 

motion compensation using either positive or negative rounding performed when the quantisation step size 
of tfie DCT coefficients is smaller than a fixed value. 

33. The method of daim 29. wherein: 

55 

assunvng that the pixel sampling period in horizontal and vertical directions are 1, the horizontal and vertical 
components of a motion vectors are integral multiples d 1/d (d being a positive integer and also an even 
number), and 
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motion compensfition using eithei positive or negative founding is pertormed when the value of d is larger than 
a fixed value. 

34. The method of claim 29, wherein intornution indicating whether positive or negative rounding was used tor motton 
5 compensation for a frame, pi eferably for the chrominance signal of a frame, is included in the encoded data of eaid 

franw. 

35. The method of claim 29 having frames called I frames coded by Intraframe coding, and P frames coded l>y Inter- 
frame coding utilising unidirectional prediction lor motion conrpensation, wherein different rounding methods are 

10 used for odd and even numbered P frames occurring along the time axis from the most recently decoded I IrBme, 
preferably for the chrominance signal of these P frames. 

36. The method of claim 29 which utilises two different nnotion compensation methods assunrvng rK)n-negative integer 
values for chrominance and luminance intensity values, allows the vertical and horizontal components of the pbcal 

16 motion vectors to take values that are not integral multiples of the pixel sampling period in the vertical and haizon- 
tal directions, calculates the intensity value for a point within a reference irnage where no pixels are present by 
means of interpolation of sampling values from neighbouring pixels, and obtains a final prediction image by aver- 
aging the two prediction images obtained by said two different motion conrpensation methods, wherein positing 
averaging is used for said averaging when negative rounding is used for said two different motion compensation 

20 methods, and negative averaging is used for said averaging when positive rounding is used for said two different 
motion compensation methods, where positive averaging rounds the results of averaging with real number opera- 
tions to the nearest integer and rounds haH integer values (0.5 added to an integer) away from zero, and negative 
averaging differs from positive averaging in that half integer values are rounded towards zera 

25 37. An image sequence coding method which utilises two different motion compensation methods assuming rwrvneg- 
ative integer values for chrominance arxJ luminance intensity values, and obtains a final prediction image by avw- 
aging the two prediction images obtained by said two different motion compensation methods, wherein both 
positing and negative averaging is used for said averaging, where positive averaging rounds the results of averag- 
ing with real number operations to the nearest integer and rounds haH integer values (0.5 added to an integer) away 

30 from zero, and negative averaging differs from positive averaging in that half integer values are rounded towards 
zera 

38. A method of coding images comprising the steps of: 

35 storing a reference image, 

synthesising a prediction image by perfornrung motion compensation between said reference image and an 
input image, and 

generating multiplexed information including information of said nx)tion vectors and informaion specifying a 
rounding method which is used for pixel value interpolation in said motion compensation. 

40 

39. A method of coding images comprising the steps of: 

storing a reference image, and 

performing motion compensation by comparing an input image arxj said reference image to estimate motion 
45 vectors and syntheslse a prediction image, 

wherein a rounding method which is used for pixel value interpolation in said nrKrtion compensation tor 
synthesising said prediction image is different from a rounding method used for synthesising a prediction 
image of said reference image. 

so 40. The method of claim 39. further comprising generating rruiltiplexed information including information of said motion 
vectors and information specifying said rounding method used for synthesising said prediction image. 

41. An image encoder corr^srising: 

55 a memory for storing a reference image. 

a block matching section to synthesise a prediction image by performing motion compensation between said 
reference image and an input image, and estimate nrwtion vectors, and 

a multiplexer to generate multiplexed information induding inlbrmation of said motion vectors and infbrmatton 
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specifying a rounding method which i& used for pixel value interpolation in said motion compensation. 

42. An image encoder comprising: 

a memory for storing a reference image which is a prey^ousty decoded Image, 

a tdock matchingi section to synthesise a prediction image by performing motion compensation between an 
input image arxi said reference image and estimate rrwtion vectors, and 

a rounding method determination device for determining the rounding method used for pixel value interpolation 
in said nx>tion compensation. 

43. A method of decoding images conprising the steps of: 

storing a refererv^e image which is a previously decoded image, 
receiving information of motion vectors, and 

perforrning nrKstton compensation to synthesise a prediction image by using said motion vectors and said ref- 
ererK:e image, 

wherein the rounding method which is used for pixel value interpolation In said motion compensation for 
synthesising said prediction image is different from a rourxfing method used for synthesising a prediction 
image of said reference image. 

44. A method of decoding image comprising the steps of: 

storing a reference image. 

receiving information including information of motion vectors arxl information specifying a rour>ding method 
which is used for synthesising a prediction image in an encoder, and 

synthesising a prediction image by performing motion compensation using said motion vectors and said refer- 
ence image. 

wherein the rourxjing method used for pixel value interpolation in said nmtion compensation Is control- 
led according to said information specifying a rounding method. 

45. An image decoder comprising: 

a memory for storing a reference image, which Is a previously decoded image, and 

a synthesiser for synthesising a prediction image by performing motion compensation using received motion 

vectors and said reference image, 

wherein said synthesiser controls the rounding method used for pixel value Interpolation In said motion 
compensation, so that the rounding method lised for synthesising said prediction image is different from that 
used for synthesising a prediction image of said reference imaga 

46. An image decoder comprising: 

a memory fa storing a reference Image, which is a previously decoded image, arxf 

a synthesiser for synthesising a prediction Image by performing motion compensation using received motion 
vectors and said reference image, 

wherein said synthesiser receives information specifying the rourxiing method used in encoding proc* 
ess and controls the rounding method used for pixel value interpolation in said motion conpensation according 
to said information specifying the rounding metfiod. 

47. The invention of any of claims 38 to 46, wherein said rounding is positive or riegativa 

48. A recording medium for recording Information of Irriages which is encoded by performing motion compensation, 
vifherein said infamation includes information of spedfying a rounding method used for pixel value interpolation in 
said motion compensation. 

49. The recordng medium of claim 48, wherein said Information specifying a rounding consists of one bit 
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